Shai Ben - David – Research Plan

The complexity of machine learning tasks can be measured along two parameters; Information and computation. Information complexity is concerned with the generalization performance of learning algorithms. Namely, How many training examples are needed to achieve certain accuracy of prediction? How fast do empirical estimates converge to the true population parameters? Etc. The computational complexity of learning concerns the computation applied to the training data in order to extract from it the learners predictions. Examining the wide variety of analytic results in the area, there seems to be a tradeoff between these two complexity parameters. Learning paradigms that enjoy computationally efficient algorithms fail to have satisfactory generalization guarantees, and vice versa. Although this phenomena is manifested across the board of machine learning theory, we are far from understanding it, let alone from being able to prove any rigorous tradeoff results. I view this question as one of the most basic and intriguing challenges in the area of computational statistics. I have been working on related issues in the past, and I plan to carry out related research in the future. Some of the topics I list below can be naturally viewed as different aspects of the above basic issue.